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Summary 

Computer  assisted  exercises  have  developed  over  the  last  ten  years  into  the  most  cost-effective  method  to 
perform  collective  headquarters  training  in  NATO.  Considerable  advance  has  been  made  to  support  the  con¬ 
cept  of  “train  as  you  fight  or  operate”  by  achieving  close  integration  between  simulation  environments  and 
command  and  control  information  systems.  Key  factors  for  the  success  of  CAXes  are  the  selected  simulation, 
the  level  of  training  and  competence  of  exercise  planners  and  support  personnel  and  the  ability  to  mediate 
effectively  between  simulation  and  CCIS  used  by  the  exercising  headquarters.  For  operational  level  exer¬ 
cises,  the  Joint  Theater  Level  Simulation  was  selected  as  the  preferred  exercise  driver.  Effective  methods  for 
exercise  planner  training  and  simulation  data  base  development  have  been  developed  combining  clearly 
defined  organisational  concepts  and  tools.  The  methods  of  training  have  been  refined  but  remain  manpower 
intensive.  Exercise  management  has  been  addressed  more  recently  and  is  showing  great  potential  for  effec¬ 
tive  formalisation  and  support.  Based  on  current  practice  and  technological  investigations,  future  avenues 
that  will  contribute  significantly  to  this  field  are  simulation  composability  through  interoperability,  advanced 
distributed  learning  and  highly  flexible  data  collection  and  analysis. 


1  Introduction 

The  mission  of  NC3A’s  CAX  project  is  to  support  NATO’s  Allied  Command  Europe  (ACE)  in  the  specifi¬ 
cation,  development,  implementation  and  evolution  of  a  training  and  exercising  organisation.  This  organisa¬ 
tion  will  make  use  of  the  most  advanced  methods  and  tools  available  to  perform  its  tasks  of  individual  and 
collective  training  and  exercising.  Hence  the  emphasis  on  assistance  by  automated  systems  in  the  prepara¬ 
tion,  conduct,  observation  and  analysis  of  training  events. 

An  evolutionary  methodology  of  systems  development  is  applied  which  relies  heavily  on  user  participation 
and  experimentation.  Methods  of  organisation  and  work  and  supporting  tools  are  developed  and  tested  in  a 
laboratory  environment.  Sufficiently  successful  prototype  capabilities  are  subsequently  applied  during  exer¬ 
cises.  This  phase  of  field  testing  is  essential  in  evaluating  capabilities  with  a  broad  user  set  under  realistic 
performance  conditions.  The  empirical  data  that  can  be  gathered  in  this  manner,  forms  the  basis  for  the 
acquisition  process  of  capabilities  that  will  meet  user  requirements  and  will  be  able  to  continue  to  evolve. 

The  CAX  development  within  ACE  has  gone  through  a  process  of  laboratory  development  from  1991  to 
1994,  followed  by  extensive  field  testing  from  1995  to  1999  and  is  currently  transitioning  into  the  acquisition 
of  an  initial  capability  through  the  creation  of  an  ACE  training  organisation  called  the  ACE  Command  and 
Staff  Training  Programme  (ACSTP).  Subsequently  the  organisational  methods  and  supporting  tools  will 
evolve  through  short  development,  test  and  acquisition  cycles. 

This  document  describes  the  practice  of  computer  assisted  exercising  that  has  been  developed  through  the 
evolutionary  process  described  above.  The  critical  factors  that  determine  the  success  of  any  simulation- 
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driven  exercising  capability  are  introduced.  The  rationale  for  selecting  the  current  model  of  choice  for  ACE 
and  the  associated  management  approach  are  discussed.  Considerable  effort  has  been  expended  on  defining 
and  refining  an  effective  exercise  preparation  methodology  and  supporting  tools.  Current  practice  is  dis¬ 
cussed.  In  addition  to  simulation  models,  effective  mediation  with  C2  systems  employed  by  staffs  during 
exercises  is  essential  in  achieving  an  environment  that  enables  “to  train  as  you  fight”.  Proven  methods  and 
approaches  will  be  highlighted.  Finally,  some  major  technological  areas  that  can  contribute  to  the  evolution 
of  a  cost-effective  CAX  capability  for  ACE  and  NATO  will  be  discussed  briefly. 


2  Overview  and  CAX  critical  success  factors 

In  planning  a  CAX,  it  is  critical  to  identify  and  describe  the  training  audience  in  terms  of  their  objectives.  A 
clear  definition  of  objectives  in  a  co-ordinated  manner  among  and  within  headquarters  is  essential  in  devel¬ 
oping  the  expectation  of  the  nature  and  of  the  results  of  the  exercise.  NC3A’s  CAX  experts  participate 
directly  in  the  planning  process  of  computer  assisted  exercises  to  ensure  that  achievable  objectives  are  set 
and  to  inform  exercise  planners,  participants  and  command  groups  of  the  capabilities  and  limitations  of 
automated  exercise  support  tools.  This  aspect  of  the  planning  process  grows  in  importance  with  the  number 
of  headquarters  involved  in  the  exercise. 

Having  decided  upon  achievable  objectives,  the  design  of  an  effective  information  systems  architecture  that 
combines  the  simulation  environment  and  the  headquarters  operating  environment  into  a  representative  in¬ 
formation  management  environment  is  critical.  Locations,  from  which  the  training  audience  will  be  operating 
and  the  tools  and  systems  that  they  intend  to  employ  to  communicate  with  their  commanding  and  subordi¬ 
nate  echelons,  need  to  be  considered.  This  aspect  will  have  a  direct  influence  on  the  nature  of  the  communi¬ 
cations  systems  that  are  used  to  enable  data  flow.  The  various  options  for  the  location  of  response  cells  to  the 
lowest  decision  level  of  the  training  audience  and  the  ability  to  provide  automated  interfaces  from  the  CAX 
environment  can  be  derived  from  this  decision.  Overall  location  decisions  constitute  an  important  cost  driver 
for  exercises  and  are  therefore  a  critical  aspect  of  the  CAX  planning  process. 

During  a  CAX,  the  real  world  is  replaced  by  a  simulated  world.  The  simulated  world’s  state,  content  and  be¬ 
haviour  are  defined  by  a  combination  of  data  and  logic.  The  data  is  contained  in  the  databases  that  form  the 
input  for  the  simulation(s)  that  contain  the  logic.  Therefore  it  is  necessary  to  ensure  a  direct  involvement 
with: 


(1)  the  development  path  and  the  developers  of  the  simulation  model(s)  that  are  used  for  CAXes 

(2)  the  definition  of  the  data  that  is  entered  into  the  simulations. 

Indeed,  the  data  describing  the  simulated  world  needs  to  be  a  sufficient  reflection  of  that  part  of  the  real 
world  that  is  simulated  to  meet  the  exercise  objectives.  Since  the  objectives  vary  from  exercise  to  exercise, 
this  is  an  on-going  task.  In  order  to  ensure  formal  participation  by  all  parties  involved  in  the  process  and 
allocation  of  sufficient  effort,  NC3A  has  introduced  the  concept  of  a  CAX  data  base  management  team.  The 
team  combines  representatives  from  the  participating  headquarters,  simulation  database  experts  and  others 
who  can  contribute  relevant  knowledge  e.g.  geo  officers,  weapon  system  specialists  etc.  Each  participating 
headquarters  formally  appoints  a  member  of  this  team.  They  are  trained  by  the  CAX  experts  in  the  data  re¬ 
quirements  and  the  logic  that  the  simulation  applies  to  the  data.  The  headquarters  members  tend  to  become 
CAX  environment  experts  and  need  to  become  a  part  of  the  CAX  management  cell  during  the  conduct  of  the 
exercise. 

From  an  information  systems  architecture  point  of  view,  it  must  also  be  noted  that  data  base  building  is  in¬ 
herently  a  distributed  activity.  The  necessary  means  in  terms  of  communication  systems,  data  construction 
and  review  tools  and  data  validation  tools  need  to  be  available  to  support  this  way  of  working. 

The  simulation  model  or  federation  of  models  that  supports  a  CAX  consumes  the  data  that  has  been  built  and 
can  portray  certain  behaviours  over  time.  It  is  a  reduction  of  the  real  world  and  therefore  will  have  limita¬ 
tions.  On  the  other  hand,  automated  simulations  are  very  powerful  in  showing  the  impact  of  the  environment, 
of  limited  perception  of  truth  and  of  logistic  constraints  on  the  ability  of  units  and  entities  to  perform  their 
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missions.  This  is  especially  true  when  the  environment  is  complex,  the  number  of  entities  represented  is  large 
and  their  nature  is  diverse.  If  provided  with  good  data,  automated  simulations  can  provide  a  consistent  be¬ 
haviour  of  entities  and  environment  over  time  that  cannot  be  achieved  through  other  means. 

Response  cells  constitute  another  critical  element  in  a  CAX.  One  or  multiple  response  cells  will  communi¬ 
cate  with  a  headquarters  staff.  Therefore  a  response  cell  must  be  viewed  as  a  set  of  people  and  systems  that 
communicate  with  the  training  audience  as  well  as  with  the  simulation  environment.  They  must  be  able  to 
provide  relevant  information  to  the  Paining  staffs  and  interpret  their  decisions  in  a  realistic  manner.  There¬ 
fore  they  need  to  combine  military  and  simulation  knowledge.  As  a  cell,  they  represent  all  the  subordinate 
echelons  below  the  Paining  audience.  Since  the  simulation  environment  may  not  be  representing  entities  at  a 
level  of  resolution  that  corresponds  to  the  level  directly  below  the  Paining  audience,  a  response  cell  must  be 
able  to  develop  plans  and  aggregate  information  over  multiple  echelons. 

A  response  cell  acting  as  a  subordinate  or  another  agency  that  exchanges  information  with  the  Paining  audi¬ 
ence  may  also  introduce  scripted  events  i.e.  not  simulated,  in  the  sequence  of  simulated  events  when  the  cir¬ 
cumstances  are  appropriate  to  meet  specific  Paining  objectives.  Ensuring  a  logical  flow  of  events  and  a  cor¬ 
rect  synchronisation  is  crucial  when  mixing  scripted  and  simulated  events.  In  view  of  the  growing  impor¬ 
tance  and  reliance  on  automated  CCIS  interfaces,  a  function  of  response  cells  is  increasingly  being  fulfilled 
by  automated  interfaces.  These  tools  will  generate  data  in  a  form  that  can  be  processed  by  the  command  and 
control  systems  used  by  the  training  audience  and  consume  suitably  formalized  tasking  from  exercising  staff. 

Effective  tools  of  the  exercise  directing  staff  are  constituted  by  the  cells  representing  the  opposing  and  other 
forces  that  play  an  active  role  in  the  simulation  database.  These  cells  are  tasked  by  the  directing  staff  to 
achieve  certain  objectives.  In  order  to  increase  exercise  cost-effectiveness,  professional  cells  employing 
powerful  order  enPy  tools  are  used  to  perform  these  functions.  Another  possibility  is  to  oppose  headquarters 
to  each  other.  In  this  case,  the  structure  of  headquarters  and  response  cell  is  mirrored.  Small-scale  exercises 
of  this  nature  have  been  executed  successfully. 

Finally,  the  core  planning  team  of  the  exercise  must  understand  the  above  factors  to  perform  its  planning 
task.  The  team  will  also  act  as  the  main  directing  staff  during  the  execution  of  the  exercise.  Therefore  it  is 
essential  that  this  team  receive  sufficient  knowledge  about  each  of  the  exercise  components.  During  the  exer¬ 
cise,  they  must  be  capable  of  performing  the  following  tasks: 

(1)  assessing  the  state  of  the  exercise  with  respect  to  the  achievement  of  exercise  objectives 

(2)  determining  effective  measures  to  steer  the  exercise  in  the  required  direction  e.g.  by  introducing 
external  constraints  like  unexpected  weather  conditions 

(3)  understanding  the  factors  that  have  conPibuted  to  the  current  state  of  the  exercise. 

These  tasks  require  a  lot  of  detailed  information  to  be  gathered  and  aggregated  and  a  deep  understanding  of 
all  CAX  elements.  Therefore  CAX  experts  on  each  of  the  elements  are  assembled  in  a  CAX  management 
team  during  the  exercise  and  are  co-located  with  the  main  directing  staff  element.  In  view  of  the  regular  turn¬ 
over  of  personnel,  the  knowledge  transfer  function  to  ensure  that  exercise  planners  arc  suitably  trained  to 
perform  their  function  is  essential. 

It  is  also  essential  to  consider  the  operational  constraints  that  will  influence  the  manner  in  which  the  CAX 
capability  will  operate.  Affordable  manning  levels,  funding  for  preparation  and  execution  should  be  scoped 
prior  to  the  selection  of  an  exercise  environment.  Particularly  the  augmentation  levels  required  for  exercise 
execution  can  be  prohibitive.  Experience  in  NATO  has  also  shown  that  professional  support  for  response 
cells  and  communications  assets  to  support  distributed  exercises  in  a  robust  manner  are  major  cost  drivers. 

From  an  infrastructure  point  of  view,  the  ability  and  need  to  achieve  networking  with  C2  systems  needs  to  be 
established.  The  ability  to  re-use  existing  communications  infrastructure  and  support  personnel  is  advisable. 
An  approach  concerning  the  provision  of  exercise-related  expertise  has  to  be  selected.  Indeed  given  the  com¬ 
plexity  of  current  exercise  environments,  the  investment  in  acquiring  subject-matter  expertise  is  non-negligi- 
ble.  When  selected,  the  personnel  should  be  maintained  in  their  role  for  a  length  of  time  that  allows  their 
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knowledge  to  be  applied  sufficiently  frequently  and  to  be  passed  on  when  personnel  transitions  to  other 
functions.  In  view  of  the  regular  rotations  of  functions  in  military  organisations,  this  last  aspect  requires  a 
well  defined  human  resource  management  approach. 


3  The  CAX  Model  of  Choice  for  Operational  Level  of  Military  Decision 
Making  and  evolutionary  approaches 

NC3A  supports  ACE  in  the  selection  of  suitable  simulation  environments  for  its  various  and  evolving  exer¬ 
cising  requirements.  NC3A  is  also  actively  involved  in  guiding  the  evolution  of  these  environments  by  cap¬ 
turing  changing  requirements,  expanding  current  capabilities  and  fostering  national  or  multi-national  efforts. 
In  this  context,  NC3A’s  CAX  project  manages  changes  to  the  Joint  Theater  Level  Simulation  (JTLS)  and 
directs  pathfinder  activities  in  the  area  of  multi-national  simulation  interoperability  for  exercises. 

Complex  simulation  models  contain  a  wide  representation  of  entities  and  logical  rules  that  describe  the 
characteristics  of  real  world  objects  and  their  behaviours.  Hence  designing  the  appropriate  data  to  feed 
simulations  and  developing  cost-effective  modes  of  utilisation  that  meet  organisational  constraints,  require 
an  investment  in  time  and  the  commitment  of  personnel  resources  to  acquire  the  necessary  expertise.  Re¬ 
using  a  simulation  environment  for  multiple  exercises  will  increase  the  return  on  this  investment.  Therefore 
SHAPE  decided  to  select  a  simulation  environment  for  its  operational-level  exercises  and  to  develop  a  small 
knowledge  base  to  manage  and  maintain  its  investment  at  NC3A. 

The  principal  factors  that  led  to  the  selection  of  the  JTLS  model  were: 

(1)  Its  ability  to  simulate  joint  and  combined  operations  including  logistics  and  intelligence  aspects 
at  a  level  of  detail  commensurate  with  the  intended  training  audience.  The  level  of  detail 
required  by  the  various  training  audiences  in  NATO  was  identified  using  the  level  of  detail  of 
information  required  by  the  lowest  levels  of  exercising  staffs. 

(2)  The  unclassified  nature  of  its  algorithms  enables  its  application  for  exercises  at  various  levels  of 
classification  in  a  NATO  and  PfP  context. 

(3)  The  lack  of  embedded  data  in  the  simulation  code  that  enables  multi-national  systems  and 
capabilities  to  be  defined  and  configured  by  the  model  operators  rather  than  by  its  developers  or 
configuration  managers. 

(4)  Its  affordability  in  terms  of  licensing  and  operating  cost. 

(5)  The  acceptance  by  the  model  management  organisation  of  SHAPE  as  a  voting  and  contributing 
member  in  the  development  of  the  simulation. 

As  a  member  of  the  JTLS  configuration  control  board,  SHAPE  participates  in  the  yearly  prioritisation  of 
trends  for  modification  and  enhancements  to  the  model.  NC3A,  as  its  technical  agent,  co-ordinates  the  inputs 
of  suggested  modifications  throughout  ACE  commands  and  introduces  them  in  the  formal  configuration 
controlled  change  process  of  the  model.  Specific  modifications  that  are  of  high  importance  to  SHAPE  and 
cannot  be  funded  by  the  configuration  control  authority  are  funded  separately.  Their  implementation  is 
managed  by  NC3A  in  close  co-ordination  with  the  configuration  control  authority. 

In  addition  to  the  management  of  the  JTLS  model,  SHAPE  has  fostered  the  co-operation  between  NATO 
nations  in  the  CAX  area  by  establishing  an  advisory  multi-national  working  group.  This  group  has  enabled 
SHAPE  to  remain  fully  informed  of  national  developments  and  has  enabled  new  operational  requirements  to 
be  presented  and  discussed.  These  interactions  have  resulted  in  the  experimentation  with  multi-national  fed¬ 
erations  of  simulations  based  on  various  simulation  interoperability  protocols.  The  collaborative  work  has 
contributed  to  the  development  of  a  NATO-wide  modelling  and  simulation  (M&S)  master  plan  and  the 
establishment  of  a  NATO  M&S  group.  A  sub-group  has  been  initiated  that  addresses  CAX  matters  specifi¬ 
cally.  This  group,  the  NATO  simulation  advisory  task  group  on  CAX,  has  subsumed  the  work  of  SHAPE’S 
multi-national  working  group  and  is  chaired  by  NC3A.  Simulation  interoperability  is  currently  perceived  as 
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the  most  cost-effective  method  of  enhancing  exercising  environments.  Particularly,  combining  proven  exer¬ 
cise  simulations  to  meet  emerging  exercise  requirements  allows  user  confidence  to  be  maintained  and  re¬ 
duces  technical  as  well  as  exercise  operational  risk.  However  it  must  be  recognised  that  this  approach  will 
require  greater  emphasis  and  resources  for  interoperability  protocol  management  and  evolution  as  well  as  for 
federation  compliance  testing.  It  also  needs  to  be  understood  that  interoperability  in  a  multi-national  context 
requires  a  sustained  and  substantial  commitment  by  the  participating  nations  and  institutions. 


4  CAX  Preparation  Methodology  and  Tools 

As  discussed  in  sections  2  and  3,  the  data  that  is  utilised  by  the  exercise  environment  is  critical  to  the  success 
of  an  exercise.  It  represents  a  considerable  investment  in  time  and  effort  on  the  part  of  the  CAX  data 
management  team  and  of  the  nations  and  headquarters  that  have  contributed  to  its  population,  verification 
and  validation.  Hence  a  formalised  process  is  required  to  maintain  the  quality  of  the  CAX  data  and  tools  are 
required  to  support  its  phases.  Paragraph  4.1  describes  a  process  that  has  been  refined  in  recent  years. 

Equally  paramount  in  the  development  a  CAX  is  to  ensure  that  headquarters  can  use  their  real  world  com¬ 
mand  and  control  systems.  This  requirement  imposes  constraints  on  CAX  databases.  It  also  implies  the  co¬ 
operation  between  diverse  organisations  managing  exercise  environment  and  day-to-day  operational  envi¬ 
ronments.  Paragraph  4.2  describes  an  approach  that  has  been  applied  successfully  in  ACE  and  that  enabled 
complex  and  sometimes  conflicting  issues  to  be  resolved  in  a  timely  manner. 

Finally,  the  success  of  an  exercise  is  highly  dependent  on  the  level  of  proficiency  of  its  response  cells  in  per¬ 
forming  their  part  as  role  players,  tasking  and  information  management  teams  and  simulation  operators. 
Paragraph  4.3  addresses  some  of  the  issues  related  to  response  cell  training  and  some  of  the  approaches  that 
have  been  pursued  with  varying  levels  of  success. 


4.1  Data  base  preparation 

As  discussed  in  section  3,  the  JTLS  model  does  not  contain  data.  Hence  the  population,  maintenance  and 
evolution  of  JTLS  databases  need  to  be  a  sustained  activity.  As  forces  re-organise  and  system  capabilities 
develop,  new  entities  need  to  be  included  in  exercise  settings  and  suitable  system  characteristic  representa¬ 
tion  in  the  simulation  database  needs  to  be  ensured.  The  task  of  data  management  is  likely  to  become  even 
more  critical  as  simulation  interoperability  starts  to  deliver  effective  federations  of  simulations. 

User  ownership  of  data  and  the  explicit  need  to  invest  time  and  effort  in  verifying  and  validating  the  quality 
of  data  used  for  exercises  are  at  the  centre  of  the  organisational  approach  that  has  been  developed  by  NC3A 
for  ACE.  Indeed  the  diverse  nature  of  NATO  forces,  systems  and  tactical  employment  concepts,  requires  the 
involvement  of  experts  of  each  nation  in  various  military  fields.  As  these  activities  require  experts  to  allocate 
time  to  perform  data  collection  and  verification  tasks  and  to  travel  to  participate  in  validation  efforts,  it  is 
necessary  for  authorities  to  recognise  the  need  for  the  investment  and  to  prioritise  associated  time  and  funds. 

For  each  exercise,  a  CAX  data  base  management  team  (DMT)  is  formally  constituted  prior  to  the  initial 
planning  conference.  The  typical  composition  of  a  DMT  is  listed  below: 

(1)  A  DMT  co-ordinator  provided  by  the  headquarters  that  has  the  lead  in  planning  the  exercise. 

The  role  of  the  co-ordinator  is: 

(a)  to  ensure  the  participation  of  all  data  providers  in  the  data  base  building  process  by  calling 
the  various  meetings  and  verifying  agreed  attendance; 

(b)  to  maintain  an  overview  of  the  contents  of  the  database  and  ensure  that  the  conditions  are 
being  created  that  will  support  the  achievement  of  stated  exercise  objectives.  Of  particular 
importance  are  the  force  ratios  that  are  established,  the  capabilities  of  selected  systems  and 
the  existence  or  lack  thereof  of  specific  high  interest  capabilities  e.g.  theatre  ballistic 
missiles. 


6-6 


(c)  to  verify  the  joint  aspects  of  the  data  base  i.e.  the  complete  force  compositions  and  the 
respective  strengths  and  weaknesses  of  services  and  sides  which  will  lead  to  the  need  to 
co-operate  during  the  execution  of  the  mission; 

(d)  to  co-ordinate  the  representation  of  non-military  entities. 

(2)  For  the  selection  of  all  air  assets  for  all  participating  sides  in  an  exercise,  the  responsibility  is 
attributed  to  a  representative  of  the  air  component  command  that  is  usually  part  of  the 
exercising  headquarters. 

(3)  A  maritime  and  marine  forces  representative  usually  provided  by  the  participating  maritime 
component  headquarters  is  responsible  for  the  selection  of  all  related  assets  including  maritime 
air  assets  and  ground-based  sensors. 

(4)  A  geographic  data  co-ordinator  responsible  for  collecting  the  data  that  is  necessary  to  develop 
the  terrain  data  base 

(5)  A  target  co-ordinator  responsible  for  all  fixed  targets  represented  in  the  exercise  database. 

(6)  A  representative  of  each  major  military  formation  that  is  represented  in  the  database  and  that 
will  be  providing  a  response  cell  during  the  exercise.  In-depth  involvement  of  the  formations  in 
the  database  building  process  ensures  that  the  capabilities  of  forces  are  suitably  represented  and 
validated.  This  approach  prevents  many  problems  that  have  been  encountered  in  the  past  when 
the  level  of  involvement  was  very  limited  and  response  cell  personnel  identified  data 
inconsistencies  in  the  final  pre-exercise  preparation  phase.  The  formation  representatives  will 
call  on  support  from  specific  subject  matter  experts  and  will  assemble  a  validation  team  that 
participates  in  the  validation  phase. 

(7)  As  most  exercises  are  set  in  a  fictitious  context,  a  country-book  representative  is  appointed  to  be 
part  of  the  data  base  management  team.  The  role  of  this  representative  is  to  ensure  consistency 
between  country  book  descriptions  and  data  base  contents.  Active  participation  by  the 
representative  is  particularly  important  in  the  early  and  final  phases  of  the  database  building 
process. 

Concerning  the  unit  database  building  aspect,  a  side-orientated  organisation  of  the  database  management 
team  has  been  abandoned  because  it  led  to  imbalances  in  side  strengths  and  capabilities,  which  were  incom¬ 
patible  with  exercise  objectives. 

The  establishment  of  a  permanent  data  base  management  has  been  suggested  to  reduce  exercise-specific  data 
base  building  activities.  ACE’s  northern  region  is  investigating  the  possibility  to  form  such  a  group.  The 
provision  of  exercise  independent  funding  for  this  task  may  be  a  challenging  issue  and  may  need  to  be  ad¬ 
dressed  from  a  more  structural  perspective.  Indeed  the  establishment  of  an  exercise  data  management  organi¬ 
sation  within  the  headquarters  structure  may  be  required  to  ensure  sustained  personnel  and  information 
systems  support. 

Over  the  last  five  years  a  process  has  been  developed  that  consists  of  five  distinct  phases: 

(1)  An  entity  design  phase,  which  covers  approximately  four  to  six  weeks  and  starts  approximately 
9  to  12  months  before  the  intended  execution  date  of  the  exercise.  Typically  this  phase  will  start 
after  an  initial  planning  conference  has  taken  place. 

(2)  2  data  collection  and  verification  phases.  Each  phase  lasts  for  about  4  to  8  weeks.  The  phases 
are  consecutive  in  time  and  start  after  the  conclusion  of  the  entity  design  meeting. 

(3)  a  data  validation  phase,  which  covers  4  to  6  weeks.  It  follows  the  conclusion  of  the  verification 
phases. 
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(4)  a  Startex  validation  phase  which  covers  1  to  2  weeks  and  starts  approximately  6  weeks  prior  to 
the  stai't  of  the  exercise.  Indeed  all  the  important  aspects  of  the  exercise  need  to  be  well  defined 
to  make  this  phase  effective. 

The  entity  design  phase  starts  with  the  formal  appointment  of  representatives  to  the  data  base  management 
team.  Following  some  introductory  briefings  on  simulation  capabilities,  the  need  is  described  to  make  a 
number  of  decisions  concerning  the  modelling  of  the  natural  environment  and  of  the  entities  that  will  popu¬ 
late  the  simulated  world.  Based  on  the  NC3A  team’s  understanding  of  the  stated  exercise  objectives,  a  pro¬ 
posed  set  of  entity  designs  is  compiled  and  distributed  to  the  DMT  members.  The  phase  culminates  with  a 
two-day  meeting,  whose  objectives  are: 

(1)  To  refine  the  main  database  building  decisions:  detailed  decision  on  geographical  area  and 
important  features,  number  of  sides,  factions  within  sides,  size  of  force  and  type  of  organisation, 
level  of  detail  of  forces  and  non-combatants,  logistic  concepts,  C2  systems  that  will  consume 
exercise  data 

(2)  To  conduct  an  initial  review  of  the  forces,  target  sets,  unit  types  and  logistic  categories  that  exist 
in  the  core  CAX  database  and  in  the  archive  of  specific  exercise  database.  These  data  sets  are 
commonly  referred  to  as  “shopping  lists”. 

(3)  To  re-visit  and  assign  the  specific  responsibilities  of  each  individual  DMT  member. 

(4)  To  review  and  amend  the  data  base  building  time  line  in  order  to  ensure  that  it  is  synchronised 
with  the  participating  headquarters  operational  planning  process  and  to  complete  the  process  in 
time  for  Startex  validation  and  for  response  cell  operator  training.  The  synchronisation  with  the 
operational  planning  process  is  critical  to  ensure  that  capabilities  that  may  be  identified  as 
essential  to  the  success  of  the  selected  course  of  action  are  actually  suitably  represented  in  the 
simulation’s  database. 

(5)  To  identify  areas  of  uncertainty  in  the  definition  of  the  exercise  setting.  Indeed  the  exact  detail 
of  exercise  requirements  may  not  be  fully  clarified  at  this  stage  of  the  exercise  planning  process. 
Ensuring  that  these  design  questions  are  identified  and  resolved  early,  is  essential  to  achieve  a 
successful  exercise. 

(6)  To  present  the  tools  and  data  exchange  procedure  that  will  be  used  to  exchange  data  between 
data  providers  and  data  base  builders. 

DMT  members,  representatives  of  host  nations  and  the  NC3A  team  of  simulation  and  database  experts  need 
to  participate  in  the  design  meeting.  In  order  to  facilitate  the  decision  making  process,  it  is  necessary  to  limit 
the  attendance  of  this  meeting. 

The  decisions  taken  during  the  entity  design  meeting  are  fully  documented  including  their  rationale.  The  re¬ 
sulting  document  serves  as  a  permanent  guideline  for  data  providers  during  the  database  building  process.  It 
is  also  very  instrumental  during  the  training  of  response  cells.  Indeed  learning  a  simulation’s  functionality 
allows  an  operator  to  interact  effectively  with  a  simulation  model.  Flowever  understanding  the  reasons  that 
have  led  to  the  representation  of  specific  entities  and  capabilities,  enable  operators  to  use  them  actively  to 
perform  their  tasks  in  a  way  that  supports  the  implementation  of  the  objectives  of  exercising  headquarters. 

Following  the  completion  of  the  entity  design  document,  a  scenario  database  is  initialised  and  distributed  to 
DMT  members  for  extension  during  the  first  data  collection  phase. 

The  success  of  the  approach  described  above  is  closely  related  to  the  development  of  a  reference  or  core  data 
base  and  of  an  exercise  data  base  archive.  The  core  database  contains  verified  and  validated  sets  of  units  and 
system  parametric  data.  The  archive  enables  terrain  and  target  databases  to  be  re-used.  Extensive  re-use  of 
data  allows  the  process  of  database  building  to  be  tailored.  Particularly  the  validation  phase  can  be  skipped 
when  new  system  capabilities  are  not  introduced  in  the  exercise  database.  Flowever  data  base  preparation 
cannot  be  compressed  too  much.  Several  feedback  phases  of  data  and  the  Startex  validation  phase  are  essen- 
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tial  elements  in  ensuring  that  exercise  requirements  and  the  means  to  achieve  them  are  understood  to  a  suffi¬ 
cient  level  of  detail  and  have  been  configured  to  achieve  an  effective  exercise. 

The  data  collection  and  verification  phases  are  organised  as  a  sequence  of: 

(1)  Data  collection  by  the  various  DMT  members 

(2)  Provision  of  collected  data  to  data  base  building  team  for  consolidation  and  interpretation 

(3)  Verification  meetings  between  data  base  building  team  and  DMT  members  to  evaluate  progress 
and  identify  remaining  open  issues. 

The  first  phase  of  data  collection  focuses  on  ensuring  that  all  relevant  entities  are  identified  and  built.  During 
the  second  phase,  the  detailed  status  of  entities  is  emphasised,  system  characteristics  are  reviewed  in  a  static 
manner.  The  consistency  with  country  books  is  also  verified  at  this  stage. 

The  product  of  these  phases  is  usually  a  database  that  contains  all  the  entities  that  were  agreed  during  the 
entity  design  phase  and  that  enables  the  exercising  of  the  outcome  of  the  operational  planning  process.  Ex¬ 
perience  has  shown  that  it  is  advisable  to  develop  a  more  robust  force  structure  for  all  sides  in  an  exercise 
setting  to  increase  the  ability  to  respond  to  late  requests  by  operational  planners  and  to  be  able  to  address 
emerging  exercise  aspects. 

In  order  to  facilitate  the  distributed  data  collection  and  verification,  simple  unit  force  structure  creation, 
editing  and  documentation  tools  have  been  developed.  They  support  DMT  members  in  selecting  and  con¬ 
figuring  the  part  of  the  force  structure  for  which  they  are  responsible.  The  tools  have  been  designed  to  enable 
the  data  base  builders  to  consolidate  the  data  base  inputs  with  limited  manual  intervention  at  their  regular 
place  of  work.  This  approach  is  considerably  more  efficient  than  a  paper  or  electronic  form-based  approach 
and  is  more  effective  because  it  reduces  the  error  associated  with  multiple  persons  manipulating  the  same 
data  item.  Similar  tools  have  been  introduced  for  building  terrain  and  static  targets. 

Having  completed  a  static  building  and  review  process,  the  DMT  can  be  assured  that  the  simulated  environ¬ 
ment  and  entities  have  characteristics  that  correspond  to  their  real-world  equivalents.  However  the  ability  of 
entities  to  interact  with  each  other  and  with  the  environment  and  the  associated  results  cannot  be  verified  by 
studying  the  characteristics  in  a  static  manner.  A  dynamic  validation  is  required  to  address  these  issues. 
During  this  phase,  the  combination  of  simulation  and  exercise-specific  data  is  subjected  to  a  set  of  detailed 
tests.  The  validation  phase  consists  of  the  following  steps: 

(1)  Definition  of  test  plans.  A  set  of  detailed  tests  has  been  developed  to  address  each  function  that 
active  entities  need  to  be  able  undertake.  Tests  that  are  relevant  in  the  context  of  the  exercise  are 
selected  and  suitably  customised  for  the  exercise  database.  As  simulations  and  entities  evolve  in 
terms  of  capabilities,  additional  tests  are  regularly  introduced. 

(2)  Assembly  of  test  teams.  DMT  members  assemble  teams  that  have  the  necessary  knowledge  to 
assess  the  results  of  the  tests.  Reference  performance  data  is  also  collected  to  support  an 
objective  calibration  of  the  simulation  database. 

(3)  An  optional  step,  which  has  been  very  successful  when  we  have  introduced  new  capabilities, 
has  been  a  pre-user  validation  session  with  simulation  developers  and  external  simulation  data 
base  experts.  The  ability  to  step  through  the  events  in  the  simulation  at  the  simulation  code 
level,  provides  a  detailed  understanding  of  the  behaviour  of  all  the  entities  that  are  involved  in  a 
test  and  allows  the  identification  of  the  important  factors  that  contribute  to  the  achievement  of  a 
particular'  outcome.  Including  external  data  base  experts  during  these  tests  enables  a  very 
effective  exchange  of  experience  and  insights  in  the  configuration  of  data  to  achieve  expected 
results. 

(4)  Conduct  of  validation  tests.  Typically  these  tests  are  organised  by  functional  area.  They  are 
conducted  in  a  centralised  manner  bringing  together  database  and  subject  matter  experts. 
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Participants  need  to  be  Pained  in  operating  the  simulation  environment.  Experience  has  shown 
that  this  requirement  is  not  always  met.  It  introduces  delays  in  the  validation  process  and 
increases  the  burden  of  analysis  by  the  introduction  of  a  higher  frequency  of  operator  error. 
Depending  on  the  progress  that  can  be  achieved  during  these  sessions  of  approximately  2  to  3 
days,  it  may  be  necessary  to  repeat  validation  tests  and  to  introduce  an  additional  session. 
Documenting  the  test  process  in  an  effective  manner  remains  a  challenging  activity  especially 
when  attempting  to  capture  reference  data  also. 

The  product  of  the  data  base  validation  phase  is  a  database  that  contains  environment  and  entities  that  corre¬ 
spond  to  their  real-world  equivalents  and  are  able  to  behave  as  such  in  the  simulated  environment. 

In  order  to  improve  the  quality  of  data,  experiments  have  been  performed  with  distributed  data  validation. 
Small-scale  versions  of  simulation  environments  have  been  provided  to  headquarters  in  conjunction  with 
Paining  and  test  plans.  Varying  levels  of  success  have  been  achieved  principally  due  to  the  amount  of  effort 
available  for  local  testing  and  due  to  the  additional  system  management  burden  placed  on  local  support 
teams.  In  order  to  alleviate  this  latter  problem,  a  port  of  the  JTLS  environment  to  a  Windows  NT  environ¬ 
ment  has  been  stalled. 

An  improved  set  of  test  definition,  execution  and  documentation  tools  are  under  development. 

The  final  phase  of  the  database  building  process  is  the  starting  situation,  or  Startex,  validation  phase.  The 
DMT  members  and  the  principal  operational  planners  are  brought  together  to  attempt  to  predict  the  flow  of 
the  exercise.  Obviously  in  a  dynamic  interactive  simulation,  the  detailed  flow  of  events  cannot  be  predicted. 
However  from  a  macro-level,  the  stalling  conditions  should  set  the  scene  for  the  probable  direction  of  the 
exercise  and  should  enable  exercise  planners  to  introduce  those  elements  that  conPibute  to  the  achievement 
of  the  exercise  objectives.  This  phase  consists  of  the  following  steps: 

(1)  Detailed  Startex  setting.  Operational  planners  are  requested  to  describe  the  intentions  and 
expected  implementation  of  their  plans  and  to  set  the  detailed  stalling  conditions  in  terms  of 
force  positioning,  attrition  levels  and  other  starting  conditions. 

(2)  Simulation  of  exercise  flow.  The  exercise  planners  and  DMT  members  confront  the  selected 
plans  to  the  intended  activities  of  other  entities  and  particularly  of  potential  adversaries  using 
the  simulation  environment  with  aggregated  order  sets  and  at  a  very  accelerated  time  speed. 
Several  iterations  are  usually  required  to  achieve  a  satisfactory  stalling  situation. 

The  validation  session  is  usually  held  at  a  central  location  or  at  the  primary  headquarters  location  and  covers 
a  three-day  period.  It  enables  a  good  assessment  of  the  exercise  stalling  conditions  and  forms  the  basis  for 
the  CAX  management  efforts  during  the  execution  of  the  exercise.  In  order  to  facilitate  the  generation  of 
aggregate  order  sets,  tools  have  been  introduced  that  allow  sequences  of  orders  to  be  defined  and  executed 
without  manual  intervention. 

The  approach  and  tools  described  above  have  been  refined  over  the  last  five  years  and  have  for  the  last  three 
years  produced  consistently  well  performing  data  bases  in  the  JTLS  environment.  Further  efficiency  im¬ 
provements  are  possible  as  well  as  effectiveness  enhancements,  however  it  needs  to  be  pointed  out  that  data 
base  development  is  a  process  that  follows  and  stimulates  the  exercise  design  process  and  as  such  cannot  be 
excessively  compressed  in  time. 


4.2  CAX-CCIS  Architecture 

An  important  operational  requirement  for  headquarters  collective  training  is  the  ability  to  employ  the  opera¬ 
tional  communications  and  information  systems  tools  that  support  the  operation  of  and  between  headquar¬ 
ters.  This  ability  is  often  referred  as  “train  as  you  fight”  or  “train  as  you  operate”.  Fulfilling  such  a  require¬ 
ment  implies  close  co-ordination  between  exercise  and  CIS  planners.  Indeed  the  CAX  and  CIS  architectures 
need  to  be  designed  and  configured  as  a  seamless  environment  for  the  exercise  to  permit  the  effective  trans- 
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fer  of  data  in  both  directions.  The  need  to  meet  security  regulations  must  also  be  addressed  and  suitable  pro¬ 
visions  need  to  be  made  to  ensure  confidentiality  of  exercise  and  real  world  data. 

In  order  to  manage  the  joint  CAX-CCIS  architecture  design  and  implementation  process,  NC3A  has  intro¬ 
duced  the  concept  of  a  CAX-CCIS  architecture  group.  The  principal  CIS  planner  for  the  exercise  leads  the 
group.  Representatives  of  all  participant  headquarters  contribute  to  the  group.  In  order  to  ensure  the  support 
of  the  CCIS  support  organisations,  they  are  requested  to  participate  actively.  NC3A  acts  for  NATO  as  the 
management  organisation  for  the  set  of  tools  that  mediate  between  simulation  environments  and  NATO 
command  and  control  information  systems. 

The  architecture  group  has  an  important  configuration  management  function  in  the  sense  that  it  needs  to  de¬ 
termine  at  an  early  stage  of  the  exercise  planning  process  which  versions  of  information  systems  will  be  used 
by  headquarters  and  which  operational  communications  networks  can  be  employed  to  support  CAX-specific 
and  operational  exercise  data.  Version  management  applies  to  the  introduction  of  new  releases  of  simulation 
environments,  mediation-ware  and  C2  applications.  The  selection  of  the  versions  needs  to  be  taken  into  ac¬ 
count  during  the  database  building  process.  Indeed  the  exercise  database  needs  to  take  into  account  any  data- 
related  constraints  that  are  imposed  by  C2  applications.  Otherwise  the  applications  will  not  be  able  to  be  ini¬ 
tialised  with  exercise  without  a  considerable  amount  of  conversion  effort.  The  data  constraints  apply  to  the 
complete  range  of  data  sets  that  are  developed  for  an  exercise  including  terrain,  targets,  units  and  system  per¬ 
formance  characteristics.  Close  co-operation  between  data  base  builders  and  C2  application  designers  is  the 
most  effective  manner  to  identify  the  complete  set  of  data  constraints. 

During  the  preparation  phase  of  an  exercise,  modifications  and  potential  enhancements  to  mediation  modules 
are  identified.  They  may  be  necessary  to  leverage  new  simulation  capabilities  or  new  C2  application  func¬ 
tionality.  Enhancements  are  implemented  if  the  architecture  team  assesses  that  there  is  sufficient  effort  and 
time  available  to  perform  the  following  steps: 

(1)  Develop  a  detailed  design  of  the  required  features.  The  operational  users  of  the  C2  applications 
as  well  as  their  technical  support  organisation  need  to  work  with  the  NC3A  development  team 
to  perform  this  task.  Inevitably  priorities  will  need  to  be  set. 

(2)  Implement  prioritised  features 

(3)  Perform  technical  testing  of  the  modified  mediation  features.  These  tests  involve  the  NC3A 
mediation-ware  development  team  and  the  technical  support  organisation  of  the  particular  C2 
application. 

(4)  Perform  operational  testing.  Using  preliminary  versions  of  the  exercise  database,  the  technically 
tested  features  are  submitted  to  an  operational  evaluation.  Typically  a  number  of  representative 
and  relevant  situations  will  be  executed  with  the  complete  simulation,  mediation  and  C2 
environment. 

(5)  Produce  and  submit  modified  security  accreditation  documentation.  This  step  needs  to  be 
scheduled  carefully  in  view  of  the  timelines  that  are  needed  by  accreditation  authorities  to  study 
new  features  and  to  grant  their  approval. 

(6)  Prepare  training  material  for  response  cells  and  operational  C2  application  users.  Indeed 
mediation-ware  may  require  specific  rules  to  be  adhered  to  in  order  to  perform  its  function.  This 
is  particularly  the  case  for  mediation  ware  that  interprets  plans  and  transforms  them  into  a  set  of 
simulation  orders. 

The  aspects  of  mediation  that  need  to  be  considered  are: 

(1)  Initialisation:  the  ability  to  transform  relevant  simulation  data  base  data  into  the  appropriate 
format  that  can  be  used  by  the  C2  application. 
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(2)  Reporting:  the  ability  to  extract  relevant  data  from  the  simulation  environment  to  provide  status 
or  event  updates.  Typically  this  form  of  mediation  is  achieved  through  messages  that  comply 
with  prescribed  formats  or  through  database  to  data  base  updates. 

(3)  Ordering:  if  the  plans  that  are  developed  by  the  headquarters  are  sufficiently  formalised  in  the 
supporting  C2  application,  the  potential  exists  to  employ  automated  mediation  to  transfer  these 
plans  into  orders  that  can  be  executed  by  the  simulation  environment. 

Currently,  the  mediation-ware  that  is  used  for  most  major  CAXes  employs  the  Joint  Theater  Level  Simula¬ 
tion  as  its  principal  data  provider  and  consumer.  In  order  to  increase  the  flexibility  and  modularity  of  the  me¬ 
diation  architecture,  a  model  independent  information  exchange  layer  called  the  Operational  Environment 
Simulation  (OES)  database  has  been  defined.  Through  a  number  of  clients,  simulation  data  is  entered  into 
and  extracted  from  the  OES  database.  The  OES  database  encapsulates  the  JTLS  model.  Hence  any  environ¬ 
ment  capable  of  generating  and  consuming  the  data  contained  in  the  OES  database  could  be  employed  to 
support  the  current  information  flow.  A  formal  report  generator,  a  Maritime  CCIS  (MCCIS)  mediator  and  a 
set  of  Initial  Combined  Air  Operations  Centre  (ICC)  modules  complete  the  mediation  process. 

The  actual  mediation-ware  that  is  employed  during  an  exercise  is  primarily  dependent  on  the  level  of  utilisa¬ 
tion  of  the  C2  applications  within  the  exercising  headquarters 


4.3  Response  cell  composition,  training  and  tools 

Response  cells  are  a  major  cost  driver  for  exercises.  They  also  constitute  a  critical  success  factor.  Hence  the 
challenge  of  the  exercise  planning  team  is  always  to  achieve  the  greatest  cost-effectiveness  by  designing 
cells  that  are  affordable  in  size,  that  have  the  appropriate  composition  and  that  do  not  require  an  excessive 
amount  of  time  for  training. 

Several  approaches  can  be  considered  with  different  merits  and  associated  cost-effectiveness: 

(1)  Full  augmentation:  cells  are  fully  manned  by  augmentation  personnel  supported  by  one  or  two 
simulation  experts.  This  approach  is  usually  applied  for  large  exercises  that  are  executed  on  a 
24-hour  basis  with  response  cells  operating  from  various  distributed  locations.  In  order  to  trans¬ 
fer  the  knowledge  that  is  necessary  to  operate  the  simulation  environment  effectively  to  the 
large  number  of  augmentation  personnel  a  three-step  approach  has  been  applied  successfully. 
The  first  step  consists  of  a  one-week  familiarisation  with  the  simulation.  Formal  briefings  and 
hands-on  sessions  are  alternated  followed  by  a  mini-exercise.  A  scenario  is  used  that  is  not 
necessarily  related  to  the  exercise.  The  second  step  is  a  two-week  train-the-trainer  session. 
During  the  first  week,  response  cell  trainers  are  instructed  in  the  detailed  operation  of  the 
simulation  using  the  database  that  will  be  employed  during  the  exercise.  During  the  second 
week,  the  focus  is  placed  on  how  these  trainers  will  convey  their  knowledge  to  the  rest  of  the 
augmentation  personnel  who  will  form  the  response  cell  during  the  exercise.  The  third  step  is 
response  cell  training  just  prior  to  the  exercise.  As  a  result  of  experience,  this  phase  comprises 
an  operator  training  phase  of  three  days,  1  day  of  operator  mini-exercise  and  two-days  of 
complete  response  cell  mini-exercise.  The  exercising  headquarters  do  not  participate  in  the 
mini-exercises  to  avoid  confusion  and  to  be  able  to  focus  on  response  cell  objectives  rather  than 
staff  goals. 

During  the  last  three  years,  the  limited  simulation  environments  that  were  installed  in 
headquarters  to  support  the  validation  of  the  exercise  database  have  also  been  employed  by 
response  cell  trainers  to  train  augmentation  personnel  from  their  headquarters.  Cells  that  were 
prepared  in  this  manner  displayed  a  considerably  better  performance  than  other  cells. 

Mixed  professional  operators  and  augmentation:  this  approach  combines  professional  operators 
who  manage  and  task  entities  with  augmentation  personnel  who  are  able  to  retrieve  relevant 
data  and  who  monitor  progress  in  the  simulation.  It  has  been  applied  very  successfully  for 
small-scale  exercises  where  a  limited  number  of  operators  are  capable  of  managing  the  entities 
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that  are  active  in  the  simulation.  Due  to  their  background,  training  of  professional  operators  is 
not  required.  Augmentation  personnel  are  principally  instructed  in  data  retrieval  tasks  and 
monitoring  tasks  environment.  A  three-day  session  prior  to  the  exercise  is  employed  to  convey 
this  knowledge.  The  final  preparation  that  precedes  the  exercise  serves  primarily  to  introduce 
the  operators  to  the  exercise  database  and  scenario  and  to  perform  a  1.5  day  operator  mini¬ 
exercise.  During  this  dedicated  mini-exercise,  the  important  phases  of  the  expected  exercise 
flow  are  executed.  Finally  a  one-day  mini-exercise  is  executed  with  the  complete  response  cell 
including  the  planning  and  reporting  element  that  does  not  need  to  interact  directly  with  the 
simulation  environment. 

It  must  be  noted  that  the  term  response  cell  used  above  applies  to  all  exercise  control  components  including 
directing  staff  and  other  forces  cells. 

Experiments  have  been  conducted  during  exercises  with  automated  tools  capable  of  performing  order  ampli¬ 
fication  and  information  aggregation  to  investigate  potential  reduction  in  training  time  and  complexity  and  to 
study  reductions  in  cell  size.  Due  to  the  technical  complexity  of  these  tools  and  limited  development  re¬ 
sources,  work  is  continuing  in  this  area  and  experimental  results  have  not  yet  been  conclusive.  Powerful  data 
visualisation  techniques,  algorithmic  approaches  and  artificial  intelligence-based  concepts  are  combined  to 
develop  these  capabilities. 


5  CAX  Conduct 


5.1  CAX-C2  mediation 

NATO  information  flows  make  extensive  use  of  formal  reports  formatted  in  accordance  with  published 
specifications  (Bi-SC  reporting  directives,  AdatP3,  local  directives).  The  Formal  report  generator  combines 
data  from  the  OES  database  with  format  definitions  to  create  text  files.  Depending  on  the  physical  imple¬ 
mentation  of  the  link  between  JTLS  and  the  HQ’s  messaging  system,  reports  can  be  emailed  to  selected  sta¬ 
tions  or  passed  to  a  military  message  distribution  system.  If  required,  the  message  header  can  comply  with 
the  ACP127  format,  enabling  distribution  based  on  Subject  Indicator  Codes. 

Report  Generation  can  be  triggered  in  three  ways: 

1.  Periodic  Reports.  The  report  is  automatically  produced  at  specified  simulation  times. 

2.  Event  based  reports.  The  report  is  produced  when  a  certain  event  happens  in  the  simulation  e.g. 
the  detection  of  an  enemy  unit 

3.  Requested  reports.  The  production  of  a  report  can  be  triggered  manually  by  entering  its  type  and 
originator  at  the  report  generator  workstation. 

It  must  be  noted  that  only  factual  information  can  be  drawn  from  the  OES  database  and  the  simulation 
model.  Report  fields  that  require  human  judgement  or  assessments  cannot  be  produced.  Response  cell  per¬ 
sonnel  need  to  add  this  data.  Also,  some  factual  information  may  not  be  available  from  the  model  such  as 
the  primary  and  secondary  task  of  a  unit,  its  echelon,  its  OPCOM/OPCON  status  etc.  The  most  effective  use 
is  for  response  cell  to  edit  the  reports  and  add  relevant  data.  If  messages  are  automatically  passed  to  the  C2 
systems,  these  missing  data  items  will  appear  blank  (or  some  default  value)  in  the  CCIS  system’s  database. 

The  JTLS-MCCIS  Mediation-ware  is  an  example  of  an  implementation  where  formatted  messages  produced 
by  the  report  generator  are  sent  to  and  automatically  processed  by  a  CCIS  system.  OTH  GOLD  contact 
reports  are  sent  at  regular  intervals  to  MCCIS  causing  the  Recognized  Maritime  Picture  (RMP)  to  update  its 
set  of  simulated  tracks  automatically. 
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In  addition  a  utility  exists  which  can  convert  a  JTLS  user  line  file  into  an  OTH  GOLD  overlay  message 
(OVLY).  Response  cell  created  graphics  can  be  passed  to  an  exercising  headquarters’  MCCIS  as  an  overlay 
to  convey  additional  information. 

ICC  is  an  integrated  C3I  environment  that  provides  information  management  and  decision  support  to  NATO 
Combined  Air  Operation  Centers  (CAOC).  The  ICC  provides  functional  support  for  the  most  critical  Air  C2 
functions  at  the  CAOC  level,  such  as  Planning  &  Tasking,  Air  Task  Order  (ATO)  and  Mission  monitoring. 

The  ICC  -  JTLS  Mediation-ware  provides  the  CAOC’s  the  possibility  to  use  their  own  C2  System  during 
JTLS  exercises.  The  mediation-ware  is  bi-directional.  Data  is  retrieved  from  ICC  into  JTLS  (tasked  mis¬ 
sions).  JTLS  data  is  inserted  and  updated  into  ICC  (e.g.  mission  reports). 

The  ICC  -  JTLS  Mediation-ware  contains  4  modules: 

(1)  ICC  Initialize  module:  This  module  extracts  relevant  data  from  the  JTLS  scenario  database  and 
translates  it  to  the  ICC  database  e.g.  aircraft  characteristics,  air  defense  sites,  radars  and  (enemy) 
targets  are  stored  in  an  ICC  exercise  database. 

(2)  ATO  compiler:  The  CAOC’s  plan  an  ATO  in  ICC.  Once  released,  the  ATO  compiler  translates 
the  ATO  from  the  ICC  format  to  JTLS  orders.  These  orders  flow  into  JTLS. 

(3)  ICC  update  module:  Once  a  compiled  ATO  is  read  into  JTLS,  data  can  be  sent  back  to  ICC. 
Mission  flight  information,  reports  and  analysis  as  well  as  status  of  squadrons,  airbases,  air 
defense  and  radar  sites  can  be  kept  updated  in  ICC. 

(4)  ICC  track  Formatter:  JTLS  Lack  data  is  transformed  and  sent  to  the  ICC  workstations  enabling 
a  Recognized  Air  Picture  (RAP)  to  be  shown  on  ICC’s  AD  API  tool.  A  link  exists  between 
tracks  and  mission  data  to  enhance  RAP  monitoring. 


Several  methods  can  be  distinguished  to  pass  information  from  the  OES  database  to  a  CCIS  system: 

(1)  Direct  database  to  database  connection.  In  this  method,  an  interface  process  copies  data  directly 
from  tables  in  the  OES  database  to  tables  in  the  CCIS  system’s  database.  This  is  done  in  the 
case  of  the  ICC  interface. 

(2)  Formatted  messages,  automatically  parsed  by  the  receiving  CCIS  system.  Data  from  the  OES 
database  is  formatted  according  to  specified  rules  and  transmitted  on  a  designated  channel 
(email,  tare,  direct  connection)  to  the  CCIS  system  where  it  is  automatically  processed.  The 
MCCIS  interface  is  an  example  of  this  method. 

(3)  Formatted  messages,  parsed  by  operator  intervention  at  the  receiving  CCIS  system.  Similar  to 
the  previous  method  but  an  operator  has  to  take  some  actions  to  absorb  the  information  in  the 
CCIS  system. 

(4)  Formatted  messages  only. 

In  addition  to  a  data  base  driven  approach  to  mediation,  NC3A  and  the  US  Defense  Modelling  and  Simula¬ 
tion  Office  (DMSO)  have  developed  a  mediation  solution  for  JTLS,  to  communicate  with  ICC,  MCCIS  and 
the  report  generator  according  to  the  specification  of  the  High  Level  Architecture  as  shown  above. 

The  mediation-ware  has  been  refined  over  the  last  4  years.  It  has  allowed  the  size  of  air  response  cells  to  be 
reduced  and  has  greatly  contributed  to  the  refinement  of  headquarters  way  of  using  C2  applications  by  ena¬ 
bling  their  utilisation  on  a  scale  and  intensity  that  cannot  be  achieved  through  other  means. 


The  impact  of  using  direct  interfaces  between  simulation  and  CCIS  is  the  need  to  have  greater  validity.  In¬ 
deed,  discrepancies  between  simulation  and  real  world  become  almost  immediately  apparent  to  the  training 
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audience  e.g.  an  automated  maritime  picture  display  is  updated  automatically  and  will  show  the  disappear¬ 
ance  of  a  ship  in  an  almost  real  time  manner.  Hence  it  must  have  a  valid  explanation.  With  the  growing  for¬ 
malism  given  to  instructions  between  headquarters  and  subordinates  in  C2  systems,  prototype  interfaces  have 
been  developed  by  NC3A  that  are  capable  of  transferring  these  orders  into  the  simulation  environment  with¬ 
out  manual  intervention.  Work  in  this  complex  area  is  on  going. 


5.2  CAX  management 

From  an  organisational  perspective,  a  method  has  been  developed  and  applied  successfully  that  turns  the 
team  that  has  prepared  the  exercise  database  into  the  CAX  management  cell  for  the  exercise.  Indeed  these 
persons  combine  a  deep  knowledge  of  the  exercise  objectives,  of  the  courses  of  action  and  expectations  of 
operational  planners,  of  the  simulation  model  and  of  the  simulation  data  that  has  been  engineered  to  support 
the  achievement  of  the  objectives.  Co-located  with  the  core  of  the  directing  staff  they  can  monitor  exercise 
progress  and  advise  on  any  immediate  or  other  important  exercise  control  decisions. 

One  of  the  tools  that  are  regularly  employed  for  exercise  control  co-ordination  is  Video  Tele-Conferencing. 
(VTC)  Following  a  formalised  cycle  of  meetings,  typically  two  per  day  for  24-hour  exercises,  this  method  of 
distributed  meeting  has  proven  to  be  effective.  The  main  factors  that  contribute  to  its  success  are  the  high 
degree  of  co-operation  that  the  tool  promotes  through  its  increased  information  content  compared  to  a  tele¬ 
phone  conference  and  the  need  to  confer  in  an  organised  and  disciplined  manner.  The  objective  of  the  VTC 
meetings  is  to  address  exercise  control  issues  from  an  exercise  operational  and  from  a  technical  perspective. 
The  leaders  of  all  the  exercise  control  cells  participate  in  the  meetings  and  contribute  their  impressions.  This 
approach  enables  a  strong  team  spirit  to  be  achieved  even  though  members  are  distributed  over  large  dis¬ 
tances. 

For  the  management  of  more  detailed  questions  that  are  raised  by  members  of  exercise  control  cells,  a  proce¬ 
dure  of  electronic  mail  exchange  with  the  CAX  management  cell  has  been  developed.  This  approach  is  to  be 
recommended  over  other  approaches  because  it  enables  the  following  essential  CAX  management  aspects  to 
be  addressed  in  a  very  efficient  manner: 

(1)  When  reporting  a  suspected  issue  or  problem,  cell  operators  have  the  ability  to  cut  and  paste 
actual  simulation  data  into  their  message.  The  CAX  management  receives  the  detailed 
information  that  it  requires  to  investigate  the  reported  problem  and  can  respond  with  an  equal 
level  detail.  Compared  to  a  telephone,  note  taking  and  third  person  input  into  a  problem 
management  tool,  this  approach  offers  great  advantages. 

(2)  Messages  can  be  organised  by  sending  cell  and  sub-organised  into  open  and  closed  items.  Most 
modern  messaging  environments  support  the  management  of  message  flows  using  custom 
folders.  Furthermore  timeliness  of  responses  can  be  tracked  and  responses  that  may  be  of 
interest  to  multiple  cells  can  be  propagated  very  easily. 

(3)  The  level  of  importance  and  criticality  can  be  indicated  in  a  message  characteristic  or  subject 
header  enabling  rapid  selection  and  retrieval  by  members  of  the  CAX  management  cell. 

(4)  Finally  CAX  management  team  members  do  not  need  to  react  in  real  time  to  incoming  requests 
in  a  way  that  is  required  when  problems  are  reported  using  telephones.  This  efficiency  gain  is 
quite  considerable. 

Although  having  instituted  an  effective  method  to  manage  and  act  upon  reports  of  suspected  problems  is 
critical,  CAX  management  can  be  greatly  improved  if  problems  can  be  anticipated,  studied  and  remedied 
before  they  become  apparent  to  the  exercise  control  cells.  Obviously  the  overall  objective  is  to  ensure  that 
problems,  either  real  or  perceived,  do  not  become  apparent  to  the  exercising  headquarters.  The  most  impor¬ 
tant  classes  of  reported  problems  are  either  related  to  parametric  data  or  to  events  being  considered  as  incon¬ 
sistent  with  exercise  objectives  or  as  an  unfair  usage  of  simulation  features.  In  order  to  address  this  problem 
domain,  NC3A  has  developed  and  experimented  with  a  set  of  tools  that  monitor  simulation  events  and  pro- 
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vide  aggregate  views  of  the  state  of  the  simulated  world.  Concerning  the  monitoring  of  critical  events,  exer¬ 
cise  experimentation  has  shown  that  perceived  problem  issues  are  often  related  to  two  areas: 

(1)  High  levels  of  attrition  over  a  very  short  period  of  time:  important  losses  to  army  formations 
over  short  periods  of  time  can  impact  the  execution  of  the  course  of  action  selected  either  by  the 
exercising  headquarters  or  by  the  exercise  control  cells  responsible  for  other  forces  than  those 
managed  by  the  exercising  headquarters.  The  CAX  management  cell  has  been  equipped  with  a 
number  of  views  that  depict  losses  over  time  and  sort  them  by  most  important  victim  over  a 
selected  time  frame.  These  tools  have  enabled  the  CAX  management  cell  to  become  aware  of 
the  impact  of  losses  more  rapidly  than  the  cells  that  are  actually  managing  the  forces. 

(2)  Attrition  to  high  value  assets:  typically  in  the  airforce  and  maritime  areas,  assets  which  are 
considered  and  perceived  to  be  highly  unlikely  to  be  the  subject  of  attrition  by  exercise  control 
personnel  are  monitored  by  class.  Attrition  is  flagged  instantly  and  associated  causal  data  is 
assembled  in  a  processed  form. 

Combined  with  an  ability  to  analyse  the  reasons  for  the  attrition,  an  assessment  of  the  causes  can  be  made 
rapidly.  Potential  courses  of  action  can  be  evaluated  and  acted  upon  before  the  problem  is  actually  raised  by 
the  cell  that  is  managing  the  affected  forces.  The  ability  to  anticipate  problems  and  to  provide  detailed  de¬ 
scriptions  of  the  causes  of  events,  contributes  greatly  to  the  stability  of  the  exercise  environment. 

It  should  be  noted  that  the  close  integration  between  simulation  and  C2  systems  increases  the  need  for  close 
monitoring  of  events  and  for  the  availability  of  a  capability  to  trace  the  causes  of  specific  events,  even  more. 

From  an  aggregate  exercise  management  perspective,  a  very  important  step  in  managing  a  CAX  effectively 
is  the  definition  of  a  scenario  flow  reference.  This  reference  data  is  built  up  and  validated  during  the  Startex 
validation  session.  An  expected  flow  in  time  and  space  for  the  exercise  is  developed  as  a  result  of  the  ses¬ 
sion.  The  flow  in  time  and  space  is  expressed  using  a  set  of  aggregate  measures,  which  are  collected  during 
the  exercises.  The  comparison  of  intended  and  actual  exercise  flow  enables  the  CAX  management  cell  to 
assess  exercise  progress  from  a  macro  perspective  and  greatly  reduces  the  tendency  to  react  to  individual 
events.  Indeed  the  intensity  of  exercises,  during  which  many  events  that  would  normally  take  place  over 
longer  periods  of  time  are  concentrated  and  during  which  carefully  laid  plans  are  sometimes  gravely 
disturbed,  can  lead  to  a  tendency  to  react  very  negatively  to  adverse  events.  The  ability  to  maintain  and 
visualise  the  “big  picture”  is  the  most  effective  tool  to  counter  such  situations. 


5.3  Observation  and  analysis 

Currently  observation  and  analysis  from  a  CAX  perspective  is  primarily  focused  on  the  simulation  environ¬ 
ment.  Co-ordinated  and  combined  observation  of  simulation  and  operational  environment  has  not  yet  been 
addressed  although  the  need  to  do  so  is  recognised.  The  tools  that  are  discussed  in  the  previous  section  and 
that  are  very  effective  for  real-time  CAX  management  are  currently  also  the  main  tools  that  are  employed  for 
after-action  analysis. 


6  Future  Needs 

The  experience  that  has  been  gained  through  exercises  and  experiments  over  the  last  ten  years  in  the  field  of 
CAXes  that  focus  on  the  operational  level  of  decision  making  has  been  discussed  in  the  previous  sections. 
Great  methodological  and  technical  progress  has  been  made.  However  requirements  continue  to  evolve  and 
expand.  At  the  same  time,  the  pressure  to  use  available  personnel  and  financial  resources  in  an  ever  more 
efficient  manner  continues  to  increase.  Together  they  pose  considerable  challenges  to  the  developers  and 
operators  of  simulation  environments.  This  section  will  discuss  some  of  the  technological  avenues  that  might 
be  considered  to  meet  these  challenges. 
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6.1  Simulation  composability 

Over  the  last  ten  years,  aggregate  simulation  interoperability  as  well  as  entity  level  simulator  interoperability 
have  developed  considerably  and  have  been  used  for  operational  exercise  support  purposes.  Many  of  the 
problems  associated  with  the  utilisation  of  networks  of  simulations  and  simulators  are  understood.  Interop¬ 
erability  protocols  for  these  two  domains  have  been  merged  in  the  specification  for  the  High  Level  Archi¬ 
tecture  (HLA).  Recent  experience  shows  a  Pend  in  combining  simulations  to  meet  specific  exercise  require¬ 
ments  rather  than  expanding  individual  simulations.  A  major  factor  in  this  trend  is  the  need  to  combine  dif¬ 
ferent  levels  of  detail.  As  aggregate  simulations  would  need  to  be  modified  considerably  to  provide  the  re¬ 
quired  level  of  detail  in  some  areas  and  detailed  entity-level  simulations  cannot  manage  the  size  and  number 
of  entities  required  for  an  aggregate  scenario,  the  temporary  combination  of  the  simulations  appears  the  most 
cost-effective  solution.  For  more  elaborate  confederations  in  which  many  different  simulations  participate, 
issues  of  data  base  preparation,  response  cell  training  and  exercise  management  need  to  be  studied  further. 
Concerning  the  area  of  preparation,  validation  of  data  that  is  used  across  simulations  and  validation  of  effects 
of  systems  needs  to  be  addressed  carefully.  From  a  response  cell  preparation  perspective,  the  training  of 
augmentation  personnel  needs  to  be  investigated  particularly  in  the  case  that  operators  may  be  interacting 
with  multiple  simulations.  Finally  managing  a  federation  of  simulations  in  a  manner  that  was  discussed  in  the 
previous  section  needs  to  be  studied.  Indeed  it  is  highly  likely  that  data  that  is  exchanged  between  simula¬ 
tions  will  not  provide  sufficient  data  to  support  the  exercise  control  process  as  discussed  in  the  previous  sec¬ 
tion. 

Anticipating  meaningful  combinations  of  levels  of  detail  and  identifying  associated  sets  of  simulations  can 
greatly  contribute  to  the  ability  to  meet  exercise  requirements  within  the  12  to  18  month  planning  cycle  of  an 
exercise.  Indeed  attempting  to  reduce  the  amount  of  time  required  by  the  proven  technical  process  of  inter¬ 
operability  development  and  testing  to  fit  within  an  exercise  planning  cycle,  should  not  be  advocated.  The 
increased  technical  risk  and  likely  problems  and  failure  during  exercises  can  only  detract  form  utilising  this 
very  cost-effective  approach. 

Further  combining  interoperable  simulations  with  C2  mediation  and  intelligent  agents  capable  of  reducing 
exercise  control  cell  personnel  requirements  appeal-  to  be  effective  avenues. 

One  of  the  key  factors  of  success  for  this  type  of  approach  is  the  commitment  of  sufficient  resources  to  the 
management  aspect  of  federations  of  simulations,  C2  mediation  and  intelligent  agents.  Indeed  the  manage¬ 
ment  of  interoperability  agreements,  the  certification  and  testing  of  federations  and  the  management  of  their 
employment  needs  to  be  properly  resourced  in  order  to  provide  longer  term  benefits. 


6.2  Advanced  Distributed  Learning 

As  discussed  in  section  4.3  training  of  exercise  control  cell  personnel  is  a  critical  aspect  for  the  success  of 
any  computer-assisted  exercise.  The  regular  turnover  of  military  personnel  requires  a  continuous  approach  to 
Paining.  As  pointed  out  in  section  2,  this  training  must  start  with  exercise  planners.  In  view  of  the  distributed 
nature  of  the  Paining  and  the  availability  of  reliable  digital  communications  between  headquarters,  the  ex¬ 
panded  usage  of  computer-based  training  tools  and  distributed  interactive  capabilities  needs  to  be  explored 
and  leveraged.  Efficiency  gains  due  to  reduced  travel  time  requirements  and  effectiveness  improvements  as¬ 
sociated  with  more  extensive  and  verifiable  training  opportunities  are  the  likely  benefits. 

The  same  technologies  can  be  applied  for  augmentation  personnel  training  although  the  ability  to  communi¬ 
cate  with  them  may  be  reduced.  Indeed  augmentation  personnel  may  not  necessarily  be  reachable  through 
secure  digital  communications  networks.  Training  across  public  networks  should  be  considered  and  investi¬ 
gated. 


6.3  Observation  data  collection  and  interpretation 

Finally  an  area  that  remains  to  be  developed  considerably  is  the  structured  analysis  of  performance  of  collec¬ 
tive  Paining  during  CAXes.  The  process  of  deriving  data  collection  requirements  from  exercise  objectives 
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and  aggregating  group  behaviour  into  representative  measures  of  performance  that  can  support  the  assess¬ 
ment  of  a  headquarters  ability  to  conduct  certain  missions  needs  to  be  improved  and  facilitated  significantly. 
In  particular  it  is  considered  essential  that  data  collection  and  correlation  inside  the  combined  simulation  and 
C2  environments  be  implemented  to  achieve  this  objective.  The  environments  need  to  be  instrumented  in 
such  a  manner  that  meta-data  can  be  collected  and  can  be  related  to  each  other  thus  enabling  events  and 
problem  situations  to  be  monitored  from  their  inception  to  their  resolution. 

The  provision  of  an  objective  methodology  and  supporting  tools  to  perform  effective  collective  performance 
measurement  is  necessary  to  support  the  development  of  training  doctrines  and  standards  for  NATO  head¬ 
quarters  and  partner  nations. 
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Managing  xha  model  and  data  as  Distaff  (1) 


%  CAX  management  cell  iri  Distaff 

•>  Acts  as  advisor  to  exercise  control  director 
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Trie  problem  management  process 

-  Perceived  problems  are  discussed  iri  PCs  or  other  cells  with  operators 

-  if  solution  cannot  be  found,  e-mail  using  template  to  CAX 
management  cell  through  JTL3  workstation 
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-  CAX  management  cell  implements  solution  arid  e-mails  confirmation 
to  problem  reporting  cell 

-  Do  riot  report  perceived  problem  to  training  audience 
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oopbJoi  apd  oopdiik  ooapabo  ? 
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Notes  for  Slide  38 


A  pictorial  representation  of  the  DiMuNDS  2000  federation  is  shown 
above. 

Ground  forces  will  be  represented  by  two  (separate)  instances  of 
KIBOWI.  Having  two  instances  of  the  STRADIVARIUS  model  is  also 
under  consideration. 

NATO  C3  Agency  acts  as  the  Federation  Manager  for  the  DiMuNDS 
2000  project,  supported  by  additional  staff  from  The  Netherlands 
Organization  for  Applied  Scientific  Research  (TNO)  and  Virtual 
Technology  Corporation  (VTC).  The  DiMuNDS  2000  federation 
management  effort  is  jointly  funded  by  NC3A  and  DMSO. 
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93500]  GojJcJlJSJDJTJS 


DavaJoprriarrj:  ra JabvaJy  obaap 

j  SaJabva  io  jayaabmarri  In  vdloaa  £> j /jj jjlIsi'J:] o JTJgj  bajarj  oombjaad  3j/j d 
ooaf  of  laid  Lid  ad  fba  furioboriaJjfy  of  oaa  or  raora  jnfo  aaobbar 

•  CorjsjdarabJa  ooordjrjabori  arid  raa/japa/oam  ovarbaad 

j  DavaJop  arid  doou/uou f  miaroparabjJjb/  apraaraama 
j  Layoui  aad  varay  dayaJopraaai  afapa 
j  Orparaaa  aad  aaaJyaa  iaaia 

•  boaforraaaoa  faaaap  aaaaaaaJ 

j  Taafa  aaad  fo  ba  davaJopad  by  oaafraJ  aafborrb/ 
j  racparaa  varajoaboa  by  oararaJ  nm'nofivj 

•  bJuJaadaboaaJ  afforf  rapuaaa  fad  arid  auafajaad  ooraraaraara 


%  SlnjuJaiJo/j  3ompc>3']bjJj£y 

■*  Canjbjrja  ajj/jukido/j  aira/j^jiha  -i/jd  apacifjc  cbaraciarlsdca  to 
acbja'/a  moro 

■*  JTL3,  J-JLA  ii/jd  a-.d3d.n5j  rad  am  don  Objaci  blodaJs 
■»  JncJuda  inia/opambiJliy  v/iih  02  appJjciidons 

*  Djsirjbuiad  Lasindncj  for  awards  a  control  uuynjanhidon  ;mj;jj/j£j 

•  j'jJaibO£jI>Jo£|J33  tO  nJ333U/3  a^ardsa  ban  am 

•»  J/jdl vidLJsiJ  Jaarniny  /naaau/amani  undamiood 

■*  Dry -intend  on  teaming  niaaau/anjani  J333  y/aJJ  dannad 
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